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Summary.  In  this  paper  we  derive  the  log  likelihoodh-function 
for  point  processes  in  terms  of  their  stochastic  intensities, 
using  the  martingale  approach.  For  practical  purposes  we  work 
with  an  approximate  log  likelihood  function  which  is  shown  to 
possess  the  usual  asymptotic  properties  of  a  log  likelihood 
function.  The  resulting  estimates  are  strongly  consistent  and 
asymptotically  normal  (under  some  regularity  conditions) .  As 
a  by-product,  a  strong  law  of  large  numbers  and  a  central  limit 
theorem  for  continuous  martingale  are  derived. 
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1.  Introduction.  The  maximum  likelihood  estimates  for  point 
process  model  have  been  occasionally  used  especially  for  Poisson 
model  (see  e.g.  Snyder  (1975)).  The  log  likelihood  function  has 
also  been  used  in  some  cases  in  a  somewhat  heuristic  manner  (see 
e.g.  Snyder  (1975),  Vere-Jone (1975) ) .  Although  it  is  expected 
that  the  maximum  likelihood  estimates  possess  the  usual  asymptotic 
properties,  to  the  best  of  our  knowledge,  there  has  not  been  a 
rigorous  proof  for  it.  In  this  paper,  we  use  some  of  the  ideas 

of  Jacod  (1975)  and  Lipster  and  Shiryayev  (1978)  on  martingale 
theory  for  point  processes  to  derive  the  log  likelihood  function, 
and  we  prove  under  some  regularity  conditions  the  asymptotic  pro¬ 
perties  of  the  log  likelihood  and  the  maximum  likelihood  estima¬ 
tors.  The  case  of  self  exciting  processes  is  of  great  interest 
since  the  log  likelihood  can  be  written  down  easily  and  hence  is 
treated  in  greater  detail. 

2 .  The  Log  Likelihood  Function  for  Point  Process. 

We  shall  be  concerned  with  multitype  point  processes,  that  is, 
point  events  of  r  different  types  randomly  occurring  along  the 
real  line.  This  process  can  be  described  by  a  multivariate  count¬ 
ing  process  N(t)  =  {N^  (t)  , .  . .  ,N  (t)  }  ,  t  e  R,  defined  on  some 
probability  space  (Q,A,P).  Here  (t)  -  N^(s)  ,  t  >  s  denotes 
the  number  of  point  events  of  type  j  which  occur  in  (s,t]  . 

By  convention  Nj(0)  =  0  .  We  shall  suppose  that  at  each  t  , 
at  most  one  event  regardless  of  its  type  can  occur.  Let  A  , 
t  s  0  be  an  increasing  family  of  sub-o  fields  such  that  N(t) 


is  A^-measurable ,  t  >  0  .  Then  (Lipster  and  Shiryayev  (1977), 


p.  239)  there  is  for  each  j  =  l,...,r  a  natural  increasing 


process  A ^  (t)  ,  called  the  compensator  of  N ^ (t)  ,  relative 

to  (At,P)  such  that  nij(t)  =  (t)  -  A ^  ( t)  t  >  0  is  a 

A^-local  martingale.  Here  we  shall  be  interested  only  in  the 

case  where  the  measure  dA.{t)dP  admits  a  density  X  .  (t)  with 

3  3 

respect  to  dtdP  .  The  process  X^ (t)  can  be  chosen  to  be 
At-predictable,  that  is  (t,w)  ->■  Xj  (t,uj)  is  measurable  with 
respect  to  the  o-field  generated  by  all  the  Afc-adapted  pro¬ 
cesses  with  left  continuous  sample  paths,  and  is  called  the 
stochastic  intensity  of  the  N(t)  process.  Intuitively  X^ (t) 
can  be  interpreted  as 


(2.1)  lim  P{  AN  . ( t)  =  1 1 A  }  /  At 

At+0  3  Z 

where  AN(t)  =  N(t  +  At)  -  N(t)  .  Indeed  if  t3  is  the  time  of 

n 

occurrence  of  the  n-th  event  of  type  j  after  the  origin,  then 
(t  a  t3)  ,  t  >  0  is  a  martingale  and  so  for  s  >  t  ,  A  e  , 


(2.2)  E[  1_  (N  .  (s  T3)  -  N  ,  (t  T3)  }]  =  E{ 


S  T  3 

a  n 


a  n 


a  n 


W  J 

J  t  TJ 
a  n 


dA. (u) } 


E[1a  1  .  X.  (u) ]du  . 

A  {u  S  T3}  3 
n 


Heuristically  if  s  =  t  +  At  and  At  is  small,  then  the  left 
hand  side  of  (2.2)  is  approximately  P (A  n  {t  s  t3}  n  {AN(t)  =  1}) 
and  the  right  Rand  side  of  (2.2)  is  approximately 


E[1a  1 


.  X(t) lit 
U  s  t3> 


Since  {t3  <  t)  e  A.  ,  we  obtain  the 
n  t 


interpretation  (2.1).  Historically  (2.1)  was  proposed  as  the 
definition  of  the  stochastic  intensity.  This  definition  requires 
the  existence  of  the  limit  in  (2.1)  and  is  equivalent  to  our 


definition  only  under  some  regularity  conditions. 

If  A  =  A  ,  the  sub  c-field  generated  by  N(s)  ,  0  -  s  *  t  , 
then  the  stochastic  intensity,  which  we  write  now  X^(t)  com¬ 
pletely  defines  the  probability  distribution  of  the  process  (Jacod 
(1975),  Lipster  and  Shiryayev  (1977),  p.  252).  Now  let  tt  be  the 
probability  such  that  the  N^(t)  ,  relative  to  tt  ,  are  independent 
Poisson  processes  with  unit  rate.  Thus,  relative  to  it  ,  the  stoch¬ 
astic  intensity  of  N^(t)  is  one.  Observe  that  the  random 
measure  {j}  x  (s,t)  ->  ft  Xj(u)du  on  {l,...,r}  x  r+  is  precisely 
the  dual  predictable  projection  in  the  sense  of  Jacod  (1975)  of  the 
random  measure  {j}  x  (s,t)  -*•  (t)  -  (s)  ,  by  the  result  of 

this  paper,  P  is  absolutely  continuous  with  respect  to  tt  on  any 
At  ,  T  >  0  ,  with  density 

=  exp  [/T  J  {Log  (t)  dN(t)  -  X ^  (t)  dt  +  dt  }]  . 

Let  now  { P q ,  0  e  Q>  be  a  family  of  probability  distributions  on 
(ft, A)  and  let  Xfl  . (t)  be  the  corresponding  stochastic  intensity 
of  the  Nj (t)  process.  Then  the  above  result  shows  that  the  log 
likelihood  function  corresponding  to  an  interval  of  observation  T0,TJ 
is,  up  to  an  additive  constant: 

T  l 

(2.3)  L_,  (9)  «  r  l  (Log  Xfl  .(t)  dN.(t)  -  Xfl  .(t)  dt}  . 

1  Oj=l  ° ' J  J  J 

Remark 

The  multitype  process  is  a  special  case  of  the  marked  point 
process  when  the  space  of  the  marks  is  just  {l,...,r}  .  There  is 
no  difficulty  to  write  down  the  log  likelihood  function  for  marked 
point  process.  Also,  it  is  not  necessary  to  suppose  the  existence 
of  the  stochastic  intensity  to  write  down  the  log  likelihood  func- 


tion;  it  suffices  that  the  measure  dA.  .  (t)  dP0  be  absolutely 

0|]  o 

continuous  with  respect  to  some  measure  dv(t)  dPQ ,  !\a  .  (t) 

e  o»] 

being  the  natural  increasing  process  of  N.(t)  relative  to  A 

1  t 

and  to  Pg  .  The  special  case  we  considered  is  convenient  for 
further  developments  concerning  the  asymptotic  properties  of  the 
log  likelihood  function. 


3 .  The  Approximate  Log  Likelihood  Function 

The  considered  process  is  defined  on  the  whole  line  although  only 
an  observation  on  [0,T]  is  available.  Denote  by  Xft  . (t)  the 
stochastic  intensity  of  the  N^(t)  process  relative  to  Pg  and 
the  sub  o-fields  Afc  ,  t  e  R  generated  by  N(s)  ,  s  <  t  .  Models 
of  point  processes  are  usually  described  in  terms  of 
Xg(t)  =  (Xg  1(t),...,Xg  r(t))  .  For  example,  the  self  exciting 

process  introduced  by  Hawkes  (1972)  can  be  defined  by 

t 

(3.1)  XQ(t)  =  ttg  +  /  g  0  ( t  -  s)  dN  ( s) 

where  an  is  a  constant  vector  and  gQ(.)  is  some  appropriate 

y  o 

matrix  function.  Thus,  it  is  desirable  to  obtain  the  log  likeli¬ 
hood  function  in  terms  of  XQ(t)  .  We  are  led  to  the  problem  of 

computing  XQ(t)  in  terms  of  XQ  . (t)  .  Now,  from  the  interpret- 

y  u ,  j 

ation  (2.1)  of  XQ  . (t)  one  can  expect  that 

y  i  d 

X„  .(t)  =  Eq{X-  .(t)  I  A.}  in  case  when  XQ  . ( t)  is  integrable. 

0,3  00*3  t  o,3 

The  rigorous  result  is 

Theorem  1.  Let  be  the  time  of  occurrence  of  the  n-th  event 

-  -  n - - - - 

of  type  j  after  the  origin.  Then  for  almost  all  t  ,  the  random 


6. 


variable  1 


<  T j |  XQ^j(t)  ijs  Pp-integrable  and 


X{t  <  t^}  xe,j(t)  E0[1{t  <  x^}X0,  j  (t)  lAt] 


almost  surely. 


Proof. 

From  (2.2),  we  have  for  all  s  >  0 

n  *  E  «j(s  „  x3)  }  -  Is  E  [1{  Tj,X  (t)]dt 
J  o  n  J 

and  hence  the  function 


(t,<d)  +  Xn(t,(D)  =  l{t  <  Tj.  (W)  X0  .  (w) 

n'  •  -I 

belongs  to  L'*'(R+  *  ft,8R+  *  A,  dt  dPg)  •  We  shall  show  that 
there  exists  a  At-predictable  process  Xn(t)  with 
Xfi(t)  =  E{Xn(t)|At>  for  almost  all  t  ,  almost  surely.  Indeed, 
there  exists  a  sequence  of  simple  functions  of  the  form 

mk 

_  Xn  v (t ,u))  =  l  Z<k)  (oo)  1  (k)  (k)  .  (t)  k  =  1,  2,  ... 

n'K  m=l  m  ^m- 1 '  m  J 

which  converges  in  L1  to  Xfi  .  For  t^^  <  t  s  tj^  ,  set 

X  ,  (t)  to  be  a  version  of  E0(z*k^  |A  }  ,  such  that  the  process 
n,k  w  m  t 

X  ,  (t)  has  left  continuous  sample  path  and  is  A  - 
n ,  k  o. 

adapted,  which  is  possible  because  of  the  martingale  property 

of  the  Efi  ( Z  A  }  ,  t*k)  <  t  <  tj;k)  .  Hence  X  .  (t)  is 
0  m  t  m-l  m  n ,  k 

/^-predictable . 

From 


lxn,k<t>  *  ’  lEe[(x„,k(t)  ‘  Xn(t,1|/'t1  1 

5  Ee(lxn,klt)  ‘  Vt)llAt' 


7. 


and  the  fact  that  X  ,  X  in  L^"  as  k  •+  °°  ,  we  get  that 

n » J'-  n 

the  sequence  X  .  ,  k  >  1  is  a  Cauchy  sequence  in  L1  and 

n  /  k 

hence  converges  to  some  A t~predictable  process  X^  which 

equals  Eg{Xn(t)  | A t }  almost  surely  for  almost  all  t  . 

Now,  from  the  definition  of  X  (t)  and  the  fact  that 

n 

{t  <  t^}  e  ,  we  have,  almost  surely 

1{t  <  Ee{Xm(t)  I  *t}  =  Vxn(t)iAt}  for  m  >  n 

and  hence  there  is  a  At~predictable  process  X(t)  such  that 


X(t)  = 


Xn  < t)  = 


t}A0, j (t) 


for  almost  all  t  ,  almost  surely. 

We  now  show  that  X(t)  is  \  ■  ( t )  .  For  this  let  A  £  A.  , 

V  f  J  t 

then  the  right  hand  side  of  (2.2)  is  equal  to 

Ee!Jt  Wu)  -  J  Ee:iA  Ee(xn(u)  |  Au)]du 

EefV„l“> )  du 

rs"T^  - 

“  Ee{1A  J  X (u)  du }  . 

Hence,  by  (2.2)  the  process  N(t)  -  X(u)  du,  t  >  0  is  a  local 
martingale,  which  gives  the  desired  result. 


Corollary.  Suppose  that  X^  j (t)  is  integrable  for  almost  all 
t  .  Then  Xn  .  (t)  =  E  { \  .  (t)  I  A.  }  almost  surely,  for  almost  all 

— - U  /  J  V  C/l  X, 

t  . 

Although  the  above  result  orovides  a  means  of  computing  X^ (t) 


8. 


in  terms  of  A0 (t)  ,  the  actual  computation  is  not  easy.  So 
we  are  led  to  approximate  A0 (t)  by  some  A0(t)  which  depend 
only  on  N(s)  ,  0  <  s  <  t  .  The  approximate  log  likelihood 
function  is  then 

T  r 

(3.2)  Lm (6)  =  /  {  I  Log  A„  . (t)  dN.(t)  -  Afl  .  (t)  dt> 

1  0  j  =  l  ° J  J  9  j 

Since  T  ^  LT(0)  depends  essentially  on  the  values  of 
A g ( t )  for  large  t  if  T  is  large,  we  would  expect  that 
is  a  good  approximation  to  LT  for  large  T  if  X0(t)  is  a 

good  approximation  to  Ag(t)  for  large  t  .  But  by  the  corol¬ 

lary  of  Theorem  1  and  the  stationarity  of  N(t)  , 

E II  Ag(t)  -  A 0  ( t )  1 1  "*■  0  as  t  °°  .  Therefore  one  could  expect 

~  A 

that  Lt  is  good  approximation  to  LT  for  large  T  if  A0 (t) 
is  a  good  approximation  to  A0(t)  for  large  t  .  We  will  make 

A 

our  assumptions  on  A0(t)  precise  later  on. 

As  example,  consider  the  self  exciting  process  when  A0  (t) 

is  given  by  (3.1).  In  order  that  Afl  . (t)  a  0  for  all  t  ,  we 

shall  assume  that  aQ  _.£  >  0,  gQ  _.£(t)  2  0  for  all  t  .  If  the 

g0  are  integrable  and  N(t)  is  of  stationary  increments  with 

ENj(l)  =  U0  j  <  00  ,  then  the  integral  in  (3.1)  exists  almost 
surely.  Indeed 


{/< 


-  s)  dN£  ( s)  }  = 


/fcg 


e,3£ 


(t  -  s) 


e,  £ 


ds 


~  P0  ,%  -0 


ge, j*lt)  dt  <  +‘ 


and  hence  the  integral  in  the  above  left  hand  side  is  finite  almost 
surely.  As  an  approximation  to  A0(t)  ,  one  might  consider 


Ag (t)  =  aQ  +  /  gQ(t  -  s)  dN(s) 


(3.3) 


which  is  evidently  At-measurable  and  would  be  a  good  approx¬ 
imation  to  Ag  ( t)  for  large  t  if  gg(t)  ■*  0  sufficiently 
fast. 


4 .  Asymptotic  Properties  of  the  Approximate  Log  Likelihood 

From  now  on,  we  shall  suppose  that  the  N(t)  process  is  of 
stationary  increments  and  metrically  transitive  in  the  sense  of 
Doob  (1953,  p.  510).  Let  S)  be  the  space  of  £5  =  (£5^,...#£5  ) 
where  the  ok  are  non-decreasing  integral  valued  functions  on 

-v 

(_oof»)  t  A  the  o-field  generated  by  the  projections 

7Tgt  :  £5  -*■  £5(t)  -  £o(s)  ,  s  <  t  ,  Pg  the  restriction  to  A  of  the 

image  of  PQ  by  the  application  N  :  u>  N  ( . )  ,  and  T,  the 
v  n 

shift  operator  (T^w)  (t)  =  w(t  +  h)  .  Then  stationarity  means 

that  conserves  the  probability  Pg  ,  that  is 

Pg (T  ^A)  =  P  (A)  ,  V  A  e  A  ,  and  metric  transitivity  means  that 

the  invariant  sets,  that  is,  those  sets  A  e  A  for  which 
T  Ja  =  A  ,  a.s.,  have  probabilities  zero  or  one.  Now,  from  the 
fact  that  N(t)  -  N(s)  =  o  N  and  that  conserves  the  prob 

ability  Pg  ,  one  can  show  that  Ag(t)  =  Ag(t)o  N  with 
X.  (t  +  h)  =  AQ(t)  o  T,  .  Hence  if  A.(t)  is  integrable 

U  Oil  U 

1  T 

lim  ^  I  A0 (t)  dt  =  E{A  (0) }  , 

T-*-oo  0 

almost  surely.  (see  Doob,  1953,  p.  515).  Here  the  expectation  i 

computed  with  respect  to  the  true  probability 

k  a 

We  suppose  in  the  sequel  that  0  c  R  and  AQ(t)  ,  A0(t) 


10. 


are  twice  continuously  differentiable  with  respect  to  6  , 

almost  surely.  We  shall  use  the  notation  x*1*  ,  x*2*  to 

0  0 


denote 

the  vector  and  the  matrix  of 

first 

and  second 

deriv- 

atives 

of  xQ  with  respect  to  0  . 

Let 

<Mt) 

be  a 

function 

of  the 

Vj(t)  '  1  -  1-2 

such 

that 

>Mt) 

is 

integrable.  Then  by  the  same  argument  as 

above 

(4.1) 

1  fT 

i-Jo  iMt)dt  EiMO)  , 

a.  s. 

(T  -*■  “) 

In  this  section,  we  are  interested  in  the  limiting  beha¬ 
vior  of  LT  (6)  ,  L,*1*  (0*)  ,  i  =  1,2  as  T  ,  0*  being 

the  true  value  of  0  .  We  have,  omitting  the  subscript  6 

A  A 

when  0  =  0*  and  putting  <1>Q  .  (t)  =  Log  AQ  .  (t)  , 

0  •  3  0  /  3 

dm j  =  dNj (t)  -  A j (t) dt: 


(4.2) 


(4.3) 


IT  rT  rT 

V0)  =  i  (  L  .(t)dN  (t)  ~  I  A  .(t)dt} 

1  j=l  Jo  Q'3  3  Jo  0,:i 

*  rT  a 

=  I  [  <J>fl  .(t)dm.(t) 

j  =  l  Jo  0  '  J  J 

rT 

+  .  (t)  A  .  (t)  -  £  (t)  }dt] 

Jo  Q’3  3  3 

Li.1*  (9)  =  l  (f  $H(t)dN.(t)  -  fT  ^X\(t)dt} 

T  j=l  J  0  Q’3  3  Jo  Q’3 

=  |  P  (t)  (A  (t)  -  L  (t)  }dt 

j  =  1  Jo  6 ,3  3  Q'3 


{J1*  (t)dm.  (t)  ] 

0*3  3 
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(4.4)  L^2)(0)  =  l  {  f  J^2ht)dN.  (t)  -  fT  X^2ht)dt} 

j=l  Jo  0,3  3  Jo  6'3 


=  I  t  f  Jfl2i(t){X.(t)  -  .(t))dt 

j=l  Jo  0/3  3  6,3 


+  f  J^2?.  (t)dm.(t) 

Jo  6'3  3 


fT  A  (1|  A  /1\  A 

-  J0  *6, j (t)*e,  j <c>  ’’•’e, j 


In  order  to  use  the  result  (4.1)  we  are  led  to  replace 


^9 ,  j (t)  '  by 


XQ^j(t)  ,  4>0  ^  j  ( t)  log  XQ^j(t),...  . 


The  following  result,  which  can  be  easily  proved,  is  quite  useful. 


Lemma  1.  Let  Efc,  xfc  be  such  that  et/xt  **  0  a£  t  -*•  «>  .  Also 
fT  ,  fT 

let  |  le^Jdyft)  <  00  and  y(T)  |  |x^|dy(t)  be  bounded  where  y 

JO  J  0 

is  a  non-decreasing  function  with  y(°°)  =  °°  .  Then 

-1  fT 

y(T)  I  e  dy(t)  0  as;  T  -»■  <»  . 

Jo 


To  obtain  convergence  results  for  the  stochastic  integrals 
with  respect  to  diru  (t)  in  (4.2)  -  (4.4),  we  will  need  the  follow¬ 
ing  result  which  is  of  independent  interest. 


Lemma  2.  Let  Mfc  ,  t  2  0  be  a  locally  square  integrable  martin¬ 
gale  with  continuous  natural  increasing  process  <M>t  .  Let  gfc  , 
t  >  0  be  a  non-decreasing  left  continuous  function  with  g^  =  »  , 

9  — £ 

such  that  <M>t  =  0(gfc  )  ,  e  >  0  ,  almost  surely  as  t  -*•  00  . 

Then  ■*  0  almost  surely  as  T  -*■<*>  . 


f 
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Proof 

Let  a  >  0  be  such  that  g  >  0  .  Let  c  be  an  arbi- 

a 

trary  positive  constant.  Define  the  stopping  time 

^  _2 

t  =  Inf  {t  :  fA  gg  d  <M>g  >  c}  , 


Set  Mt  =  MtAT  '  and 


-1 


tAT  -1 


Zt  -  4  gs  dMs  -  4  dMs 


The  process  Zt,  t  >  a  is  a  martingale  with  natural  increasing 
process 

tAT 

<Z> .  =  /  g  d<M>  sc,  t  s  a 
t  a  s  s 

2 

Since  EZt  <  c  ,  t  s  a  ,  we  know  (Doob,  1953,  p.  354,  361)  that 

almost  surely  Za+  ,  Zfc_  ,  Zfc+  ,  t  >  a  ,  Z^  exist  and  Zfc  is 

(k) 

bounded  on  any  finite  interval.  Set  t)  =  a  +  (T  -  a)  i/k  and 
write 

k 

gT  ZT  =  E  g  z  m  +  g  fk\  fk\  “  z  (kl^ 


i=l  t 


(k) 


.  (k)  (k) 

-i-1  fci 


*(k>  l%(k)  ".(k) 

ti_l  tt  ti.! 


=  f  Z<k>  dg  +  /T  gik)  dz 
0  a  r 


where  Z^k)  =  z  (k) .  gj.k)  =■  9 


|S1 


for  tfkj  <  t  <  tfk) 
l-l  i 


By  the  Lebesque  dominated  convergence  Theorem  and  the  proper- 
:y  of  stochastic  integral,  almost  surely  as  k  -*■  00 


T- 


(k) 


T- 


4  zt  *  4  zt*  a*t 
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and 


t:*r  «*  *  /:  .*  ■  /:  * 


-  MT  -  Ma 


Therefore 


T- 


9t1(mt  '  "a>  =  ZT  -  a;  V  dgt 


T- 


=  ZT  -  a  -  g-^"1)  Zoo  -  g^1  /  <Z^  -  Zm)  dgt 

Since  gT  +  °°  as  T  ■+  °°  ,  from  Lemma  1  ,  we  get  g~^"  (M,p  -  M&) 

and  hence  9t^j  ■*  0  »  almost  surely  as  T  -*■  °°  . 

Now,  we  have  for  all  t  on  the  set 

00  2 

{t  =  °°}  =  { J  go  d<M>  <  c}  . 

a  S 

Since  c  is  arbitrary,  we  obtain  9^1mt  0  almost  surely  as 

T  -»•  °°  on  the  set 

J°°  7 

g“  d<M>s  <  «} 


2-e 


By  assumption  <M>  s  const,  g  ,  for  all  s  >  b  ,  for  some 

s  s 


s  ~  Js 

b  s  a  ,  almost  surely.  So 


/OO  00 

gg2  d<M>s  <  const.  j^<M> 


«>  -2/  (2-e ) 

s  d<M> 


=  const.  £m>  x 
b 


oo  -2/  ( 2-e) 


dx  <  oo 


SI 


nee  the  image  of  the  measure  d<M>g  by  the  application 


s  -*■  <M>g  is  just  the  Lebesque  measure.  Hence,  almost  surely 


/ 


00  -2 

g  d<M>  <  oo 

a  vs  s 


_  o 

since  gg  is  bounded  on  [a,b]  .  The  proof  is  completed, 


14. 


Theorem  2.  Suppose  that 

(i)  P{A„  4  (t)  >  c}  =  0  for  some  c  >  0  (c  is 
U  /  J  -  - 

independent  of  t  by  stationarity)  and  £  . (t)  -  X  . (t)  , 

.  ,  °  /  3  o ,  ] 

£  ^ j  (t)  -  (t)  ,  i  =  1,2  tend  to  zero  almost  surely  as 

t  -*■  oo  . 


(ii)  An  (t)  is  integrable ,  and  <$>„  4  (t)  and  the 
0,]  -  -  - 

elements  of  <J>gM  (t)  ,  q ^  j  ( t )  are  square  integrable  with 
respect  to  the  measure  A  (t)dP-. 

Then  almost  surely  as  T  -»•  » 


T  1  L  (0)  -  E[  l  <p  (0)  A  (0)  -  A.  .  (0)  ]  =  A  (0 ) 
X  j  =1  U  9  J  J  ”  /  J 


T-1  L^1)  (6*)  •*  0 


T-1  L<2)  (0*)  -  -E{  l  6^  (OU^  (0)  ' 4>  .  (0)  =  -J 
A  j=l  3  3  3 


(1) 


(1) 


Proof 


Consider  the  right  hand  side  of  (4.2).  By  (i) 


+e,j<t)xj<t>  '  xe,j(t)  ’  *e, j (t)xe , j (t)  '  xe,j<t>  +  °(t» 


where  in  this  proof  o(t)  denotes  a  quantity  tending  to  zero 

almost  surely  as  T  -*■  60  .  By  lemma  1  and  the  fact  that 

k 

•  (t)  A  .  (t)  is  integrable  since  <|>0  •  ( t)  ( A  .  ( t) )  is  square 
integrable,  we  obtain  from  (4.1)  that  T  ^  times  the  last  term 
of  the  extreme  right  hand  side  of  (4.2)  tends  to  A (6)  almost 
surely  as  T  -*•  °°  .  On  the  other  hand,  itk  (t)  is  a  martingale 
with  natural  increasing  process  given  by  d<m^>fc  =  A^(t)dt  . 
Therefore 


15. 


M .  =  f  .  (t)  dm.  (t) 

J  #  g  v  ,  j  J 

is  also  a  martingale  with  natural  increasing  process 

<M  >  =  f  $2  (t)  X  .  (t) dt 

J  t  Q  0  ,  J  J 

If  <Mj>t  =  °(t)  almost  surely  as  t  ->■  »  t  then  by  lemma  2, 
Mj  t/T  -*•  0  and  hence  T  1  LT(0)  -*■  A  (6)  almost  surely  as 
T  +  »  .  Since  from  (i) 


.(t)X.(t)  =  <j>n  ^(t)X.(t)  +  <j>  .  (t)  X.  (t)o(t)  +  X  .  ( t)  o  (t) 

^  f  3  J  f  j  J  ,  3  J  3 

and  therefore  by  (4.1)  and  Lemma  1,  M.  /T  ->  E{ 4>^  .  (0)X.(0)} 

3  t 1  “ ,  ]  j 

almost  surely  as  T  "*  °°  ,  we  obtain  the  result 


-1  ( l  \ 

The  proof  for  the  convergence  of  T  L^,  '  (6*)  and 
-1  (2 1 

T  L,p  ( 0 * )  uses  the  same  idea.  We  have  from  (i) 


$  jL)(t)  {X  j  (t)  -  ^(t)  =  { <J)  ^ 1 5  (t)  +  <})!1)  (t)o(t)  +  o  ( t )  }o  ( t) 

=  (}):a)  (t)o(t)  +  o  ( t) 

$j2)(t){Xj(t)  -  Xj(t)}  =  {x]2)  (t)/X.  (t)  -  (t)  (t) ')o(t) 

=  Uj2)  (t)  +  Xj2)  (t)o(t)  +  (t)o(t) 

+  $ll)  (t)  4,  j15  (t)  'o(t)  +  o(t)  }o(t) 

=  4>  (t)o  <t)  +  xj2)(t)o(t)  +  4>^1}  (t)o(t)  +  o(t) 

Since  Xj(t)  is  bounded  below,  from  (ii)  ,  <j>!^  (t)  ,  X?^  (t)  , 

i  *  1,2  are  integrable.  By  lemma  1  and  (4.1),  the  first  term 
of  the  extreme  right  hand  side  of  (4.3),  (4.4),  divided  by  T 
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tend  to  zero  as  T  . 

Finally  by  a  similar  argument  as  above,  the  second  terms 
in  the  extreme  right  hand  side  of  (4.3)  and  (4.4)  are  T  o(T)  and 
the  last  term  in  the  right  hand  side  of  (4.4)  divided  by  T 
converge  to  -J  almost  surely  as  T  -*•  °°  .  The  proof  is  com¬ 
pleted. 

Remark 

Condition  (i)  of  the  theorem  is  introduced  for  convenience. 

The  result  might  hold  under  weaker  conditions.  In  fact,  it 
suffices  that 

_T  T 

kfQ  {$j<t)  *  Aj(t))dt  ,  |  40fj(t)  -  4>6j  .(t)  }X.(t)dt  , 

T 

if  (t){L(t)  -  X  .  ( t)  }  dt  ,  i  =  1,2, 

VO  ]  3  3 

ui11  (t)  l  !1}  (t)'A  .  (t)  -  4)  !x)  (t)^^15  (t)'*.  (t))dt 

g  J  J  J  J  J  J 

tend  to  zero  almost  surely  as  T  -►  <»  and 

fj0  $0fj(t)X.(t)dt  ,  Q  (t)||  2X.(t)dt  ,  i  -  1,2 

are  bounded  almost  surely,  and  condition  (ii)  to  obtain  the  result. 

Condition  (i)  is  also  not  very  restrictive.  In  case  of  the 
self -exciting  process  (3.1)  with  g  0  ,  then  x  ^(t)  is 

O  K  tJ  r  J 

bounded  below  by  a  a0  .  ,  which  we  assume  to  be  strictly  positive. 

o ,  3 

If  we  also  assume  that 

gfl(t)  =  sup 1 1  gfl(t  +  h)|| 
hs  0 


is  integrable,  then 
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A  r°  r 

!!  A.  (t)  -  \  (t)|j  <  const./  |jg  (t  -  s)||  l  dN(s) 

y  a  J  -00  j  =  l 

r°  -  r 

^  const.  I  g  (t  -  s)  £  dN(s) 

J  -oo  j  =  l 

where  the  last  integral  is  almost  surely  finite  since  it  has 

a  finite  expectation  and  converges  to  zero  as  t  »  by  the 

monotonous  convergence  Theorem.  In  the  same  way,  if  gg  (t) 

is  twice  differentiable  with  respect  to  0  with  derivatives 

satisfying  the  same  condition  as  gg (t)  as  above,  then 

(t)  -  (t)||  ,  i  =  1,2  ,  tend  to  zero  almost  surely  as 

0  ,  D  0  r  3 

t  -*•  oo  . 

-k 

Consider  now  the  asymptotic  distribution  of  T  LT(0*)  . 

From  (4.3),  we  expect  that  the  above  is  asymptotically  distributed 
like 

T  r 

(4.5)  Z_  =  T ~*  [  l  <b<1)  (t)dm.  (t) 

T  J  0  j=l  ]  3 

The  asymptotic  distribution  of  Z^  can  be  obtained  from 
the  following  result,  which  is  of  independent  interest. 


Lemma  3.  Let  Mt,At,t  >  0  be  a  square  integrable  martingale 
with  natural  increasing  process  <Mfc>  satisfying  d<M>t 


=  X  dt  . 


Suppose  that  there  is  a  semi  group  of  shift  operator  Tg,s  s  0  , 
conserving  the  probability  and  metrically  transitive,  such  that 


,-l 


h  At  -  Vh  '  <Mt-  V°  Th  -  Vh  -  Ms+h  •  T  *  "  - 


T  *5Mt  is  asymptotically  normal  with  zero  mean  and  variance  EX^ 


Proof . 

Let  nT  be  integers  such  that  =  T/n^,  ■*  0  as  T  "*■  00 
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Set 


Y 


T 


9 


j 


M(  j-1)  AT5 


Then  from  the  result  of  Durett  and  Resnick  (1978),  as  T  -*■  °°  , 

n„ 


M  =  T-i?  M.  +  y  Y_  , 
T  0  T, ] 


is  asymptotically  normal  with  zero  mean  and  variance  EXg, 
provided 


n„ 


(i)  E{YT,j  !A(j-l)AT}  EX0  in  Probability 


n„ 


(ii)  l  E[Yf  .  lr„2  >  e>]  -  0  ,  e  >  0  . 
j=l  T'=>  tYT#j  > 


To  verify  (i) ,  observe  that  the  sum  in  (i)  is  equal  to 

i  4  x‘T)  at  -  <«>T/T  +  i  /0  «*««  -  xt)  at 

where  x£T*  =  E(xt|AjAT)  for  j  A^  <  t  <  (j  +  1)  AT  .  Clearly 
Xfc+h  =  Xfco  T^  and  E(^t+h| Ag+h)  =  E(XtlAs)oTh  .  Therefore,  as 

at  0  , 


Sup  E|X*T)  -  X.  |  <  Sup  E  |  E  (X.  |  A1  )  -  X.  I  ->0 
Os  t<T  ^  1  0<u<AT  1  X_U  1 

Note  that  Xfc  is  Afc  -  measurable  because  this  process  is  predic¬ 
table.  Thus,  the  sum  in  (i)  differs  from  <M>T/T  by  a  term 
tending  to  0  in  the  mean  and  since  <M>T/T  -*•  EXq  by  the  metric 
transitivity  of  Th  ,  we  obtain  the  result. 

To  verify  (ii) ,  observe  that  the  Y  .,  j  =  l,...,n  have 

>  j 

2 

the  same  distribution.  Put  £(t)  =  -  Mq  ,  we  are  led  to  verify 


that: 
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VE(YT,! 


{Y 


]  =  A, 


-1 


T,1 


>  e} 


eCUAt) 


U(at) 


>  eT} 


1 


tend  to  zero  as  T  -*■  00  .  This  is  true  if  At  o  sufficiently 
slow.  Indeed,  C (A)  ,  0  ^  A  <  1  being  a  positive  sub-martingale 
is  uniformly  integrable  (Doob,  1953,  p.  359),  that  is: 


a(T)  =  0<a<iECC(A)  1{C(A)  >  £T}3  "  0  * 

as  T  -*■  00  .  So  all  we  have  to  do  is  to  choose  nT  such  that 
a(T)/AT  0  as  T  00  . 

We  now  show  that  the  difference  between  T^  LT(6*)  and 
ZT  ,  defined  by  (4.3),  (4.5)  tend  to  zero  in  probability  as 

T  -*■  00  . 


Lemma  4 .  Let  Mfc,  t  >  0  be  a  locally  integrable  martingale 

with  continuous  natural  increasing  process  <M>t  .  Let  hfc  >  0  , 
2 

if  <M>T/hT  -*•  0  in  probability  as  T  "*■  00  ,  then  so  is  • 

Proof 

Let  e  >  0  .  Define  the  stopping  time  to  be  the  value 

of  s  such  that  <M>g  =  eh^  .  Then  as  T  **•  03  , 


since 


P{MT«T  *  V 

PtoT  5  T}  -  P{<M>T  i  Eh^} 

e(m2t«t>  -  E<MV„T  s 


-*■  0 


0  .  On  the  otherhand 


E<M> 


and  hence  by  Tchebycheff  inequality 

P{|MT/hT|  >  6}  <  PtM^  +  mt1  +  c/62 


from  which  the  result  follows. 


Theorem  3.  Under  the  condition  of  Theorem  2  and  suppose  that 
.  f  T  A... 

(i)  T-^  I  Q)1  ‘  (t)  {L  (t)  -  X  •  (t)  }dt  -*•  0 

J  0  J  3  J 

_L 

in  probability  as  T  -*  00  .  Then  T  ^  L^ ( 6  * )  is  asymptotically 
normal  with  zero-mean  and  covariance  matrix  J  given  in  Theorem  2. 


Proof . 

Let  a 


be  a  vector  and  ZT  be  given  by  (4.5).  Then 
,  T  >  0  is  a  martingale  with  natural  increasing 


process 


Jt  r 

l  .... 

0  j=l  J  3 

since  (Lipster  and  Shiryayev,  1977,  p.  269) 


<M  >  fc  = 


(s)}  2X.  (s)  ds 


2M  dM  +  Y  (AM  ) 
s"  s  s<t  S 


H 

-  f  2M  dM  +  l  {a* (1)  (s)  }2  dN.(s) 

JO  S‘  S  j=l  j  D 

=  f  C  2M  dM  +  T  {a'4>!1)  (s)  )2  dm.(s)]  +  <M>. 

Jo  S"  8  j=l  3  3 


where  the  first  term  of  the  last  expression  is  a  martingale  and 
the  second  is  a  natural  process.  By  Lemma  3,  a'zT  is  asympto¬ 
tically  normal  with  zero  mean  and  variance  a'ja  .  On  the  other 
hand,  by  (i)  and  Lemma  4  with 

dMfc  *  Ej  (t)  -  4>  f1*  (t)  }dmj  (t)  ,  hfc  =Vt  ,  we  see  that 

T"’,*L^1^  (8*)  -  ZT  -*•  0  in  probability  as  T  -+■  00  .  The  result 
follows. 


Remark 


Condition  (i)  of  Theorem  3  is  satisfied  if,  for  example. 


E  ||(j) !  ^  ( t) 1 1  2  is  bounded  and 


(4.6) 


r°°  a 

JolEU: 


.  (t)  -  Xj (t)  \2}h  dt  <  +«> 


since  by  Schwartz  inequality,  the  first  absolute  moment  of  the 
expression  in  (i)  is  bounded  by 
.T 


f  (E||  J*1}  (t)||  2  E | X  .  ( t)  -  X  (t)  j  2} ^  dt 

Jo3  3  3 


In  case  of  the  self  exciting  process  with  X  ^ (t)  ,  X . (t) 

given  by  (3.1),  (3.3)  we  have 


..^(t  -  s)dNt(s) 


X.(t)  -  x.(t)  =  l  f  g,-k 

3  J  k  J-° o 

r  0  r  0 

=  l  {J  9jk(t  -  s)Xk(s)ds  +j  9jk(fc  “  s)dmk(s)) 


Denote  by 

..  A 


the  L  norm,  by  the  triangular  inequality 


Mt)  -  Mt)ll2  5  l  f  m  1  g  jK  ( t:  -  s)  i  II  Xk  ( s)  !l  2  ds 

k  ^  * 


il  f°  9jk(t  "  s>  E{VS)  }ds]i5 


Note  that  as  in  the  proof  of  Theorem  3,  the  martingale  Mg 
defined  by  dM  =  [.g,.  (t  -  s)dm.  (s)  has  the  natural  increasing 

S  js.  JK  K 

process  given  by  d<M>g  =  Ik  g2R(t  -  s)*k(s)  ds  .  Therefore, 
the  process  X,  (t)  being  stationary 


r, 


^(t)  -  *  j  ( t)  1 1  2dt 


<  const. 


I  j f°  lj  lgjk(s)  Ids  +  { g2jk(s)ds),53dt 


Observe  that 


(4.7) 


Ooo 

J 


9  jk  (s)  j  ds  < 


j: 


t|gjk(t)  jdt 


and,  by  Schwartz  inequality,  for  any  positive  function 
(0,°°)  which  integrates  to  one 


on 


g^k(s)ds};5dt  <  { 


CO  x  -CO 

J  o  h(t)  J  t 


g jk (s> ds  cat}*5 


j; 


(tgfk(t)/h(t)}dt 


Take  h(t) 


—  1— (V 

t  ,  a  >  0  .  Then  (4.6)  holds  if  the  right  hand 


size  of  (4.7)  and 
one  can  show  that 

„<U 


^.2+a  g2  (t)dt  are  finite.  In  the  same  way 
0  3* 

i  DU)  (t)  -  X jX)  (t)||  2  0  as  t  -*•  oo  if  the 


'jk 


(t)  are  integrable  and  square  integrable.  Hence  E||  <j>  ! ^  (t)||  2 


are  bounded  if  the  above  conditions  hold  and  X^Ct)  are  bounded 
below.  Thus  condition  (i)  of  the  Theorem  is  not  restrictive. 


5.  Asymptotic  Properties  of  the  Maximum  Likelihood  Estimator 

We  are  interested  in  the  asymptotic  properties  of  the  esti¬ 
mate  0T  ,  which  maximize  L^,  in  0  .  We  shall  use  this  general 
result  for  which  the  proof  is  quite  standard. 

Theorem  4 .  1.  Let  AT  be  a  random  function  on  0  c  R  , 

satisfying 

(i)  At(0)  -+  A  ( © )  almost  surely  as  T  -*•  ®  ,  with  A  being 
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lim  sup  Sup  {A  (0')  -  A  (6)}  0  a.s. 

T-*«=  e'eU{e) 

as  the  neighborhood  u(9)  of  8  shrinks  to  8  . 

Then  any  0T  realising  the  maximum  of  on  a  compact 

C  of  0  containing  0*  ,  converges  almost  surely  to  0*  as 
T  ->  oo. 


2.  Suppose  that  0*  is  an  interior  point  of  0  and  A  admits 

continuous  first  and  second  derivatives  with  respect  to  0  , 

denoted  by  the  vector  A^^  and  the  matrix  A^,2^  ,  satisfying 

(2) 

(iii)  As  T  -*  oo  ,  AT  (0*)  -*  -J  in  probability  and 
^ T  A^^  (0)  is  asymptotically  normal  with  zero  mean  and  covariance 
matrix  J  . 

(iv)  For  every  e  >  0 

lim  inf  P{|A^2)(0)  -  A^,2)  (8*)||  <  e  ,  V0cU(0*) 
rp“>*00 

increases  to  1  as  the  neighborhood  U(0*)  of  0*  shrinks  to 

0*  . 

Then  ©T  of  1)  is  asymptotically  normal  with  mean  0*  and 


covariance  matrix  T-1  J-'*'  . 

Moreover,  if 

k 

6m  is  T  -consistent. 

that  is  the  distributions  of 

(e  -  e*) 

are  tight,  then 

eT  -  (eT  -  A^,2)  (0t)~1A^1)  (8t)  }  ■*  0  in  probability  as  T  -»■  «>  . 


We  apply  the  above  results  with  AT  =  T_^LT  •  BY  Theorem  2, 
condition  (i)  is  satisfied  except  the  continuity  of  A ( 0 )  ,  which 
we  shall  assume.  To  see  that  0*  realises  the  maximum  of  A(0)  , 

write 

r  , 

A (8)  -  A  (8* )  =  £[E  Log { A „  .  (0) *7  (0)  }A . (0)  +  A  (0)  -  .(0)3 

*— T  t)  r  ]  J  J  J  '-'/J 


rf 
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and  note  that  log  x  ^  x  -  1  with  equality  if  and  only  if 
x  =  1  »  we  get  A ( 0 )  i  A(0*)  with  equality  if  and  only  if 
X e  j  ( 0 )  =  Aj(0)  almost  surely,  implying,  by  stationarity 
Afi  .  (t)  =  A.(t)  almost  surely.  If  the  parametr ization  is 
such  that  for  8'  ?  6  ,  XQ^(t)  is  not  equal  to  XQ.  (t) 
for  all  t  ,  almost  surely,  then  8*  is  the  unique  maximum 
of  A  . 

Theorems  2  and  3  show  that  condition  (iii)  is  satisfied. 

So  all  we  need  is  to  verify  conditions  (ii)  and  (iv) .  This 

would  require  rather  strong  assumptions  on  A„(t)  and  Xn(t) . 

9  0 

A  sufficient  set  of  assumptions  is 

AO :  For  any  compact  C  of  0  ,  there  is  a  c  >  0 

such  that  XQ  _■  (t)  >c  almost  surely  for  all  +■ 

- - -  v ,  j  - — - ' 

all  0  e  C  . 

Al:  For  any  compact  C  of  0  ' 

A 

lim  Sup  |Afl  .(t)  -  Afi  . (t) I  =  0  a.s. 
t^oo  0€C  9 '3  9 '3 

E  {Sup  4>?  (t)  }  A  .  (t)  <  » 

0  eC  6 ' 3  ^ 

A2 :  For  some  compact  neighborhood  U  of  0*  , 

lim  Sup  IjX*1*  (t)  -  X^(t)i|  =  0  a.s.,  i  =  1,2, 
t->-oo  0  cU  0  '  3 

E[{Sup  \\^2\ ( t) 1 1  2)  X  .  (t)  ]  <  - 
ecu  ,3  3 

E{Sup  ||  X<2)  (t)|!  >  <  » 
p  e  u  c  •  J 

Theorem  S.  Under  the  assumptions  AO,  Al  condition  (ii)  of 
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condition  (iv)  of  Theorem  4  is  satisfied. 


Proof . 

Let  U (0 )  be  a  compact  neighborhood  of  8  .  Then  from 
(4.2)  , 

-1  _i  r  fT  A 

Sup  T  L-  (0")  <TX£  [1  {  Sup  <p  ,  .  (t)  }dN .  (t) 

0^CU(0)  j  =  l  J  0  8"eu(0)  ,3  3 

e  T  a 

-  I  {  Inf  Xft  .  (t) }dt]  . 

J  0  (fcUC©) 


By  the  same  argument  as  in  the  proof  of  Theorem  2,  the  above 
right  hand  side  is  seen  to  converge  almost  surely  as  T  -*■  °°  to 
r 

l  CEC{  Sup  $  ,  .  (t)}X.(t)3  -  E{  Inf  X.  . (t)  } D 
j=l  0  (0)  6  '3  3  0  "eU  (0)  0,3 


By  the  monotonous  convergence  theorem,  as  U(0)  shrinks  to  0 
the  above  expression  converges  to  A (8)  and  hence 


lim  sup  {  Sup  T  ^lt(8')}  -*■  A(8)  a.s.  , 

T-h»  0  'eU  (0) 

which  gives  the  result. 

To  verify  condition  (iv)  of  Theorem  4,  from  (4.4)  observe 
that  T-1|L^2^  (8)  -  l*r£2^  (9*)  I  is  bounded  for  all  0  e  U  by 

T'1  l  [  fT{SuP  ||  (t)  -  <j>!2)  (t)j|}dN  (t) 

j=l  J  0  0eU  0,3  3  3 

+  f  {Sup  ||X^2)  (t)  -  \\2)  (t)||  }dt] 

JO  0eU  0,3  3 


Again,  by  a  similar  argument,  the  above  expression  converges 
almost  surely  as  T  -*■  00  to 
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\  E  Sup  ||4>i2),(t)  -  <})f2)  (t)||  >X  (t) 

j=i  e£u  0,3  3  3 

+  e{suP ||  xi2).  (t)  -  x (2)  (t)  || } 
e«u  ,3  3 

Note  that  we  have  used  the  fact  that 

{Sup  ||  (t)  -  <Mt)||}Vwt)  ,  Sup  ||Xfl  .(t)  -  X  ( t)  1 1 

0eU  '3  3  3  0eu  0,3  3 

are  square  integrable  and  integrable  which  follows  easily  from 
A2.  Again,  by  the  monotonous  convergence  Theorem,  the  above 
expectations  converge  to  zero  as  U  shrinks  to  0*  .  The  proof 
is  completed. 
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